chore(appsec): classify Error Tracking audit sites by darccio · Pull Request #5466 · DataDog/dd-trace-go

darccio · 2026-09-28T20:53:55Z

Status

Classify 16 of the 19 AppSec audit sites as ineligible for Error Tracking reporting:

caller API use while AppSec is disabled or without required request instrumentation
user-provided blocked-response template paths
native-library and host compatibility failures
expected limits on the request path: the global trace rate limit (DD_APPSEC_TRACE_RATE_LIMIT) and the per-request WAF event capacity limit
Remote Config failures returned to the backend through apply statuses

Three actionable sites remain intentionally unclassified:

duplicate internal WAF action-handler registration
unexpected AppSec startup failure
SwapRootOperation failure in onRCRulesUpdate: when a poll contains only ignored or ASM_FEATURES products, statuses is empty, so the failure is only logged and cannot be reported through an apply status

These sites stay unclassified on purpose; this PR does not hide them with exclusion directives. Startup timing is not a blocker for adopting them later — reports before telemetry.StartApp capture their stack eagerly and replay; the risk is ring-buffer eviction, not a bad stack trace (see internal/README.md). The classification convention and eligibility rules are documented in internal/appsec/README.md.

This branch starts from main and is not stacked on another migration PR.

Validation

go test ./appsec ./internal/appsec/... ./instrumentation/appsec/... -count=1
go vet ./appsec ./internal/appsec/... ./instrumentation/appsec/...
go run ./internal/telemetry/log/analyzer/cmd ./appsec/... ./internal/appsec/... ./instrumentation/appsec/... (make lint/errlog)
git diff --check

chatgpt-codex-connector · 2026-09-28T20:59:33Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-01T08:45:38.753790Z	`37629d4`	Manual request
🔒 Security Review	✅ Completed 2026-10-01T08:46:02.483477Z	`37629d4`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

datadog-prod-us1-6 · 2026-09-28T21:00:11Z

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 25.00%
• Overall Coverage: 71.39% (-0.13%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 37629d4 | Docs | View more details | Give us feedback!}

pr-commenter · 2026-09-28T21:38:13Z

Benchmarks

Benchmark execution time: 2026-10-01 09:17:47

Comparing candidate commit 37629d4 in PR branch session/errtrack-appsec with baseline commit e80f19e in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 340 metrics, 1 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:BenchmarkContextWithSpan/detach-no-snapshot

unstable execution_time [-1.598ns; +1.576ns] or [-5.509%; +5.433%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

scenario:BenchmarkOTLPTraceWriterFlush

darccio · 2026-09-29T06:06:47Z

@codex review

chatgpt-codex-connector · 2026-09-29T06:08:59Z

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 10c4f9fa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-29T06:09:51Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 10c4f9fa92

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10c4f9fa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

darccio · 2026-09-29T07:22:26Z

@codex review

chatgpt-codex-connector · 2026-09-29T07:24:51Z

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 10c4f9fa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-29T07:25:13Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 10c4f9fa92

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

- Document the errtrack:ignore classification convention in internal/appsec/README.md, per internal/appsec/AGENTS.md. - Correct the AddEvents rate-limit rationale: op.limiter is the global trace rate limit, not a per-request capacity limit. - Remove the exclusion on the SwapRootOperation failure in onRCRulesUpdate: when a poll only contains ignored or ASM_FEATURES products, statuses is empty and the failure is only logged, so it stays actionable in the audit.

darccio · 2026-10-01T08:35:53Z

@codex review

chatgpt-codex-connector · 2026-10-01T08:38:41Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: f06b5bd732

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f06b5bd732

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Startup timing is not a reason to leave a site unclassified: reports before telemetry.StartApp capture their stack eagerly and replay once telemetry starts. The risk is ring-buffer eviction, not a bad stack trace. Align the README guidance with internal/README.md.

darccio · 2026-10-01T08:42:57Z

@codex review

chatgpt-codex-connector · 2026-10-01T08:45:37Z

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 37629d4c7f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-01T08:46:00Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 37629d4c7f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chore(appsec): start Error Tracking eligibility review

6738578

darccio marked this pull request as ready for review September 28, 2026 20:59

darccio requested a review from a team as a code owner September 28, 2026 20:59

chore(appsec): classify ineligible Error Tracking sites

10c4f9f

darccio changed the title ~~chore(appsec): adopt explicit Error Tracking reporting~~ chore(appsec): classify Error Tracking audit sites Sep 28, 2026

darccio marked this pull request as draft September 28, 2026 21:49

darccio marked this pull request as ready for review September 29, 2026 06:37

chatgpt-codex-connector Bot reviewed Sep 29, 2026

View reviewed changes

Comment thread internal/appsec/appsec.go

Comment thread internal/appsec/emitter/waf/context.go Outdated

Comment thread internal/appsec/remoteconfig.go Outdated

chatgpt-codex-connector Bot reviewed Oct 1, 2026

View reviewed changes

Comment thread internal/appsec/README.md Outdated

Conversation

darccio commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Status

Validation

Uh oh!

chatgpt-codex-connector Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

datadog-prod-us1-6 Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Unstable benchmarks

scenario:BenchmarkContextWithSpan/detach-no-snapshot

Known flaky benchmarks

Uh oh!

darccio commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

darccio commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

darccio commented Oct 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 1, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

darccio commented Oct 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 1, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

darccio commented Sep 28, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Sep 28, 2026 •

edited

Loading

datadog-prod-us1-6 Bot commented Sep 28, 2026 •

edited

Loading

pr-commenter Bot commented Sep 28, 2026 •

edited

Loading